Computationally-Efficient Parallel Implementation of Combustion Chemistry in LES/PDF Computations

Large scale combined Large-Eddy Simulation (LES)/Probability Density Function (PDF) parallel computations of reactive flows with detailed chemistry involving large numbers of species and reactions are computationally expensive. Among the various techniques used to reduce the computational cost of representing chemistry, the three approaches in widest use are: (1) mechanism reduction, (2) dimension reduction, and (3) tabulation. In addition to these approaches, in large scale parallel LES/PDF computations, we need strategies to distribute the chemistry workload among the participating CPUs to reduce the overall wall clock time of the computations. Here we present computationally-efficient strategies for implementing chemistry in parallel LES/PDF computations using in situ adaptive tabulation (ISAT) and x2f mpi – a Fortran library for parallel vector-valued function evaluation (used with ISAT in this context). To test the strategies, we perform LES/PDF computations of the Sandia Flame D with chemistry represented using the 16-species ARM1 reduced mechanism. We present three parallel strategies for redistributing the chemistry workload, namely (a) PLP, purely local processing; (b) URAN, the uniform random distribution of chemistry computations among all CPUs following an early stage of PLP; and (c) P-URAN, a Partitioned URAN strategy that redistributes the workload only among partitions or subsets of the CPUs. We show that among these three strategies, the P-URAN strategy (i) yields the lowest wall clock time, which is within a factor of 1.5 of an estimate for the lowest achievable wall clock time; and (ii) shows acceptable weak-scaling up to 9,216 cores.